1 Data preparation

1.1 Outline

  • Load scripts: loads libraries and useful scripts used in the analyses; all .R files contained in scripts at the root of the factory are automatically loaded

  • Load data: imports datasets, and may contain some ad hoc changes to the data such as specific data cleaning (not used in other reports), new variables used in the analyses, etc.

1.2 Load packages


library(reportfactory)
library(here)
library(rio) 
library(tidyverse)
library(incidence)
library(distcrete)
library(epitrix)
library(earlyR)
library(projections)
library(linelist)
library(remotes)
library(janitor)
library(kableExtra)
library(DT)
library(cyphr)
library(chngpt)
library(lubridate)
library(ggpubr)
library(ggnewscale)

1.3 Load scripts

These scripts will load:

  • all scripts stored as .R files inside /scripts/
  • all scripts stored as .R files inside /src/

These scripts also contain routines to access the latest clean encrypted data (see next section).


reportfactory::rfh_load_scripts()

1.4 Load clean data

We import the latest NHS pathways data:


x <- import_pathways() %>%
  as_tibble()
x
## # A tibble: 147,168 x 11
##    site_type date       sex   age   ccg_code ccg_name count postcode nhs_region
##    <chr>     <date>     <chr> <chr> <chr>    <chr>    <int> <chr>    <chr>     
##  1 111       2020-03-18 fema… 0-18  e380000… nhs_bar…    35 rm13ae   London    
##  2 111       2020-03-18 fema… 0-18  e380000… nhs_bed…    27 mk454hr  East of E…
##  3 111       2020-03-18 fema… 0-18  e380000… nhs_bla…     9 bb12fd   North West
##  4 111       2020-03-18 fema… 0-18  e380000… nhs_bro…    11 br33ql   London    
##  5 111       2020-03-18 fema… 0-18  e380000… nhs_can…     9 ws111jp  Midlands  
##  6 111       2020-03-18 fema… 0-18  e380000… nhs_cit…    12 n15lz    London    
##  7 111       2020-03-18 fema… 0-18  e380000… nhs_enf…     7 en40dy   London    
##  8 111       2020-03-18 fema… 0-18  e380000… nhs_ham…     6 dl62uu   North Eas…
##  9 111       2020-03-18 fema… 0-18  e380000… nhs_har…    24 ts232la  North Eas…
## 10 111       2020-03-18 fema… 0-18  e380000… nhs_kin…     6 kt11eu   London    
## # … with 147,158 more rows, and 2 more variables: day <int>, weekday <fct>

We also import demographics data for NHS regions in England, used later in our analysis:


path <- here::here("data", "csv", "nhs_region_population_2018.csv")
nhs_region_pop <- rio::import(path) %>%
  mutate(nhs_region = str_to_title(gsub("_"," ",nhs_region)))

nhs_region_pop$nhs_region <- gsub(" Of ", " of ", nhs_region_pop$nhs_region)
nhs_region_pop$nhs_region <- gsub(" And ", " and ", nhs_region_pop$nhs_region)
nhs_region_pop
##                  nhs_region variable      value
## 1                North West     0-18 0.22538599
## 2  North East and Yorkshire     0-18 0.21876449
## 3                  Midlands     0-18 0.22564656
## 4           East of England     0-18 0.22810783
## 5                    London     0-18 0.23764782
## 6                South East     0-18 0.22458811
## 7                South West     0-18 0.20799797
## 8                North West    19-69 0.64274078
## 9  North East and Yorkshire    19-69 0.64437753
## 10                 Midlands    19-69 0.63876675
## 11          East of England    19-69 0.63034229
## 12                   London    19-69 0.67820084
## 13               South East    19-69 0.63267336
## 14               South West    19-69 0.63176131
## 15               North West   70-120 0.13187323
## 16 North East and Yorkshire   70-120 0.13685797
## 17                 Midlands   70-120 0.13558669
## 18          East of England   70-120 0.14154988
## 19                   London   70-120 0.08415135
## 20               South East   70-120 0.14273853
## 21               South West   70-120 0.16024072

Finally, we import publically available deaths per NHS region:


dth <- import_deaths() %>%
  mutate(nhs_region = str_to_title(gsub("_"," ",nhs_region)))

#truncation to account for reporting delay
delay_max <- 21

dth$nhs_region <- gsub(" Of ", " of ", dth$nhs_region)
dth$nhs_region <- gsub(" And ", " and ", dth$nhs_region)
dth
##     date_report               nhs_region deaths
## 1    2020-03-01          East of England      0
## 2    2020-03-02          East of England      1
## 3    2020-03-03          East of England      0
## 4    2020-03-04          East of England      0
## 5    2020-03-05          East of England      0
## 6    2020-03-06          East of England      1
## 7    2020-03-07          East of England      0
## 8    2020-03-08          East of England      0
## 9    2020-03-09          East of England      1
## 10   2020-03-10          East of England      0
## 11   2020-03-11          East of England      0
## 12   2020-03-12          East of England      0
## 13   2020-03-13          East of England      1
## 14   2020-03-14          East of England      2
## 15   2020-03-15          East of England      2
## 16   2020-03-16          East of England      1
## 17   2020-03-17          East of England      1
## 18   2020-03-18          East of England      5
## 19   2020-03-19          East of England      4
## 20   2020-03-20          East of England      2
## 21   2020-03-21          East of England     11
## 22   2020-03-22          East of England     12
## 23   2020-03-23          East of England     11
## 24   2020-03-24          East of England     19
## 25   2020-03-25          East of England     26
## 26   2020-03-26          East of England     36
## 27   2020-03-27          East of England     38
## 28   2020-03-28          East of England     28
## 29   2020-03-29          East of England     43
## 30   2020-03-30          East of England     45
## 31   2020-03-31          East of England     70
## 32   2020-04-01          East of England     62
## 33   2020-04-02          East of England     64
## 34   2020-04-03          East of England     80
## 35   2020-04-04          East of England     71
## 36   2020-04-05          East of England     76
## 37   2020-04-06          East of England     71
## 38   2020-04-07          East of England     93
## 39   2020-04-08          East of England    111
## 40   2020-04-09          East of England     87
## 41   2020-04-10          East of England     74
## 42   2020-04-11          East of England     91
## 43   2020-04-12          East of England    101
## 44   2020-04-13          East of England     78
## 45   2020-04-14          East of England     61
## 46   2020-04-15          East of England     82
## 47   2020-04-16          East of England     74
## 48   2020-04-17          East of England     86
## 49   2020-04-18          East of England     64
## 50   2020-04-19          East of England     67
## 51   2020-04-20          East of England     67
## 52   2020-04-21          East of England     75
## 53   2020-04-22          East of England     67
## 54   2020-04-23          East of England     49
## 55   2020-04-24          East of England     66
## 56   2020-04-25          East of England     54
## 57   2020-04-26          East of England     48
## 58   2020-04-27          East of England     46
## 59   2020-04-28          East of England     58
## 60   2020-04-29          East of England     32
## 61   2020-04-30          East of England     45
## 62   2020-05-01          East of England     49
## 63   2020-05-02          East of England     29
## 64   2020-05-03          East of England     41
## 65   2020-05-04          East of England     19
## 66   2020-05-05          East of England     36
## 67   2020-05-06          East of England     30
## 68   2020-05-07          East of England     33
## 69   2020-05-08          East of England     33
## 70   2020-05-09          East of England     29
## 71   2020-05-10          East of England     22
## 72   2020-05-11          East of England     18
## 73   2020-05-12          East of England     21
## 74   2020-05-13          East of England     27
## 75   2020-05-14          East of England     26
## 76   2020-05-15          East of England     19
## 77   2020-05-16          East of England     26
## 78   2020-05-17          East of England     17
## 79   2020-05-18          East of England     25
## 80   2020-05-19          East of England     15
## 81   2020-05-20          East of England     26
## 82   2020-05-21          East of England     21
## 83   2020-05-22          East of England     13
## 84   2020-05-23          East of England     12
## 85   2020-05-24          East of England     16
## 86   2020-05-25          East of England     25
## 87   2020-05-26          East of England     14
## 88   2020-05-27          East of England     12
## 89   2020-05-28          East of England     17
## 90   2020-05-29          East of England     15
## 91   2020-05-30          East of England      9
## 92   2020-05-31          East of England      8
## 93   2020-06-01          East of England     17
## 94   2020-06-02          East of England     14
## 95   2020-06-03          East of England     10
## 96   2020-06-04          East of England      7
## 97   2020-06-05          East of England     12
## 98   2020-06-06          East of England      4
## 99   2020-06-07          East of England      9
## 100  2020-06-08          East of England      5
## 101  2020-06-09          East of England      4
## 102  2020-06-10          East of England      7
## 103  2020-06-11          East of England      0
## 104  2020-06-12          East of England      5
## 105  2020-06-13          East of England      2
## 106  2020-06-14          East of England      0
## 107  2020-03-01                   London      0
## 108  2020-03-02                   London      0
## 109  2020-03-03                   London      0
## 110  2020-03-04                   London      0
## 111  2020-03-05                   London      0
## 112  2020-03-06                   London      1
## 113  2020-03-07                   London      1
## 114  2020-03-08                   London      0
## 115  2020-03-09                   London      1
## 116  2020-03-10                   London      0
## 117  2020-03-11                   London      7
## 118  2020-03-12                   London      6
## 119  2020-03-13                   London     10
## 120  2020-03-14                   London     14
## 121  2020-03-15                   London     10
## 122  2020-03-16                   London     18
## 123  2020-03-17                   London     25
## 124  2020-03-18                   London     31
## 125  2020-03-19                   London     25
## 126  2020-03-20                   London     44
## 127  2020-03-21                   London     50
## 128  2020-03-22                   London     54
## 129  2020-03-23                   London     64
## 130  2020-03-24                   London     87
## 131  2020-03-25                   London    113
## 132  2020-03-26                   London    130
## 133  2020-03-27                   London    130
## 134  2020-03-28                   London    122
## 135  2020-03-29                   London    147
## 136  2020-03-30                   London    150
## 137  2020-03-31                   London    181
## 138  2020-04-01                   London    202
## 139  2020-04-02                   London    190
## 140  2020-04-03                   London    196
## 141  2020-04-04                   London    230
## 142  2020-04-05                   London    195
## 143  2020-04-06                   London    198
## 144  2020-04-07                   London    219
## 145  2020-04-08                   London    238
## 146  2020-04-09                   London    206
## 147  2020-04-10                   London    170
## 148  2020-04-11                   London    177
## 149  2020-04-12                   London    158
## 150  2020-04-13                   London    166
## 151  2020-04-14                   London    144
## 152  2020-04-15                   London    142
## 153  2020-04-16                   London    139
## 154  2020-04-17                   London    100
## 155  2020-04-18                   London    101
## 156  2020-04-19                   London    103
## 157  2020-04-20                   London     95
## 158  2020-04-21                   London     95
## 159  2020-04-22                   London    109
## 160  2020-04-23                   London     77
## 161  2020-04-24                   London     71
## 162  2020-04-25                   London     58
## 163  2020-04-26                   London     53
## 164  2020-04-27                   London     51
## 165  2020-04-28                   London     43
## 166  2020-04-29                   London     44
## 167  2020-04-30                   London     40
## 168  2020-05-01                   London     41
## 169  2020-05-02                   London     40
## 170  2020-05-03                   London     36
## 171  2020-05-04                   London     30
## 172  2020-05-05                   London     25
## 173  2020-05-06                   London     37
## 174  2020-05-07                   London     37
## 175  2020-05-08                   London     29
## 176  2020-05-09                   London     23
## 177  2020-05-10                   London     26
## 178  2020-05-11                   London     18
## 179  2020-05-12                   London     18
## 180  2020-05-13                   London     16
## 181  2020-05-14                   London     20
## 182  2020-05-15                   London     18
## 183  2020-05-16                   London     14
## 184  2020-05-17                   London     15
## 185  2020-05-18                   London      9
## 186  2020-05-19                   London     13
## 187  2020-05-20                   London     19
## 188  2020-05-21                   London     12
## 189  2020-05-22                   London     10
## 190  2020-05-23                   London      6
## 191  2020-05-24                   London      7
## 192  2020-05-25                   London      9
## 193  2020-05-26                   London     12
## 194  2020-05-27                   London      7
## 195  2020-05-28                   London      8
## 196  2020-05-29                   London      7
## 197  2020-05-30                   London     12
## 198  2020-05-31                   London      6
## 199  2020-06-01                   London     10
## 200  2020-06-02                   London      7
## 201  2020-06-03                   London      6
## 202  2020-06-04                   London      8
## 203  2020-06-05                   London      3
## 204  2020-06-06                   London      0
## 205  2020-06-07                   London      4
## 206  2020-06-08                   London      5
## 207  2020-06-09                   London      2
## 208  2020-06-10                   London      7
## 209  2020-06-11                   London      5
## 210  2020-06-12                   London      2
## 211  2020-06-13                   London      2
## 212  2020-06-14                   London      0
## 213  2020-03-01                 Midlands      0
## 214  2020-03-02                 Midlands      0
## 215  2020-03-03                 Midlands      1
## 216  2020-03-04                 Midlands      0
## 217  2020-03-05                 Midlands      0
## 218  2020-03-06                 Midlands      0
## 219  2020-03-07                 Midlands      0
## 220  2020-03-08                 Midlands      3
## 221  2020-03-09                 Midlands      1
## 222  2020-03-10                 Midlands      0
## 223  2020-03-11                 Midlands      2
## 224  2020-03-12                 Midlands      6
## 225  2020-03-13                 Midlands      5
## 226  2020-03-14                 Midlands      4
## 227  2020-03-15                 Midlands      5
## 228  2020-03-16                 Midlands     11
## 229  2020-03-17                 Midlands      8
## 230  2020-03-18                 Midlands     13
## 231  2020-03-19                 Midlands      8
## 232  2020-03-20                 Midlands     28
## 233  2020-03-21                 Midlands     13
## 234  2020-03-22                 Midlands     31
## 235  2020-03-23                 Midlands     33
## 236  2020-03-24                 Midlands     41
## 237  2020-03-25                 Midlands     48
## 238  2020-03-26                 Midlands     64
## 239  2020-03-27                 Midlands     72
## 240  2020-03-28                 Midlands     89
## 241  2020-03-29                 Midlands     92
## 242  2020-03-30                 Midlands     90
## 243  2020-03-31                 Midlands    123
## 244  2020-04-01                 Midlands    140
## 245  2020-04-02                 Midlands    142
## 246  2020-04-03                 Midlands    124
## 247  2020-04-04                 Midlands    151
## 248  2020-04-05                 Midlands    164
## 249  2020-04-06                 Midlands    140
## 250  2020-04-07                 Midlands    123
## 251  2020-04-08                 Midlands    186
## 252  2020-04-09                 Midlands    139
## 253  2020-04-10                 Midlands    127
## 254  2020-04-11                 Midlands    142
## 255  2020-04-12                 Midlands    139
## 256  2020-04-13                 Midlands    120
## 257  2020-04-14                 Midlands    116
## 258  2020-04-15                 Midlands    147
## 259  2020-04-16                 Midlands    102
## 260  2020-04-17                 Midlands    118
## 261  2020-04-18                 Midlands    115
## 262  2020-04-19                 Midlands     92
## 263  2020-04-20                 Midlands    107
## 264  2020-04-21                 Midlands     86
## 265  2020-04-22                 Midlands     78
## 266  2020-04-23                 Midlands    103
## 267  2020-04-24                 Midlands     79
## 268  2020-04-25                 Midlands     72
## 269  2020-04-26                 Midlands     81
## 270  2020-04-27                 Midlands     74
## 271  2020-04-28                 Midlands     68
## 272  2020-04-29                 Midlands     53
## 273  2020-04-30                 Midlands     56
## 274  2020-05-01                 Midlands     64
## 275  2020-05-02                 Midlands     51
## 276  2020-05-03                 Midlands     52
## 277  2020-05-04                 Midlands     61
## 278  2020-05-05                 Midlands     58
## 279  2020-05-06                 Midlands     59
## 280  2020-05-07                 Midlands     48
## 281  2020-05-08                 Midlands     34
## 282  2020-05-09                 Midlands     37
## 283  2020-05-10                 Midlands     42
## 284  2020-05-11                 Midlands     33
## 285  2020-05-12                 Midlands     45
## 286  2020-05-13                 Midlands     39
## 287  2020-05-14                 Midlands     37
## 288  2020-05-15                 Midlands     40
## 289  2020-05-16                 Midlands     34
## 290  2020-05-17                 Midlands     31
## 291  2020-05-18                 Midlands     34
## 292  2020-05-19                 Midlands     34
## 293  2020-05-20                 Midlands     36
## 294  2020-05-21                 Midlands     32
## 295  2020-05-22                 Midlands     27
## 296  2020-05-23                 Midlands     34
## 297  2020-05-24                 Midlands     19
## 298  2020-05-25                 Midlands     26
## 299  2020-05-26                 Midlands     33
## 300  2020-05-27                 Midlands     29
## 301  2020-05-28                 Midlands     27
## 302  2020-05-29                 Midlands     20
## 303  2020-05-30                 Midlands     20
## 304  2020-05-31                 Midlands     21
## 305  2020-06-01                 Midlands     20
## 306  2020-06-02                 Midlands     21
## 307  2020-06-03                 Midlands     23
## 308  2020-06-04                 Midlands     15
## 309  2020-06-05                 Midlands     21
## 310  2020-06-06                 Midlands     20
## 311  2020-06-07                 Midlands     16
## 312  2020-06-08                 Midlands     15
## 313  2020-06-09                 Midlands     17
## 314  2020-06-10                 Midlands     14
## 315  2020-06-11                 Midlands     13
## 316  2020-06-12                 Midlands      8
## 317  2020-06-13                 Midlands      1
## 318  2020-06-14                 Midlands      2
## 319  2020-03-01 North East and Yorkshire      0
## 320  2020-03-02 North East and Yorkshire      0
## 321  2020-03-03 North East and Yorkshire      0
## 322  2020-03-04 North East and Yorkshire      0
## 323  2020-03-05 North East and Yorkshire      0
## 324  2020-03-06 North East and Yorkshire      0
## 325  2020-03-07 North East and Yorkshire      0
## 326  2020-03-08 North East and Yorkshire      0
## 327  2020-03-09 North East and Yorkshire      0
## 328  2020-03-10 North East and Yorkshire      0
## 329  2020-03-11 North East and Yorkshire      0
## 330  2020-03-12 North East and Yorkshire      0
## 331  2020-03-13 North East and Yorkshire      0
## 332  2020-03-14 North East and Yorkshire      0
## 333  2020-03-15 North East and Yorkshire      2
## 334  2020-03-16 North East and Yorkshire      3
## 335  2020-03-17 North East and Yorkshire      1
## 336  2020-03-18 North East and Yorkshire      2
## 337  2020-03-19 North East and Yorkshire      6
## 338  2020-03-20 North East and Yorkshire      5
## 339  2020-03-21 North East and Yorkshire      6
## 340  2020-03-22 North East and Yorkshire      7
## 341  2020-03-23 North East and Yorkshire      9
## 342  2020-03-24 North East and Yorkshire      8
## 343  2020-03-25 North East and Yorkshire     18
## 344  2020-03-26 North East and Yorkshire     21
## 345  2020-03-27 North East and Yorkshire     28
## 346  2020-03-28 North East and Yorkshire     35
## 347  2020-03-29 North East and Yorkshire     38
## 348  2020-03-30 North East and Yorkshire     64
## 349  2020-03-31 North East and Yorkshire     60
## 350  2020-04-01 North East and Yorkshire     67
## 351  2020-04-02 North East and Yorkshire     74
## 352  2020-04-03 North East and Yorkshire    100
## 353  2020-04-04 North East and Yorkshire    105
## 354  2020-04-05 North East and Yorkshire     92
## 355  2020-04-06 North East and Yorkshire     96
## 356  2020-04-07 North East and Yorkshire    102
## 357  2020-04-08 North East and Yorkshire    107
## 358  2020-04-09 North East and Yorkshire    111
## 359  2020-04-10 North East and Yorkshire    117
## 360  2020-04-11 North East and Yorkshire     98
## 361  2020-04-12 North East and Yorkshire     84
## 362  2020-04-13 North East and Yorkshire     94
## 363  2020-04-14 North East and Yorkshire    107
## 364  2020-04-15 North East and Yorkshire     96
## 365  2020-04-16 North East and Yorkshire    103
## 366  2020-04-17 North East and Yorkshire     88
## 367  2020-04-18 North East and Yorkshire     95
## 368  2020-04-19 North East and Yorkshire     88
## 369  2020-04-20 North East and Yorkshire    100
## 370  2020-04-21 North East and Yorkshire     76
## 371  2020-04-22 North East and Yorkshire     84
## 372  2020-04-23 North East and Yorkshire     63
## 373  2020-04-24 North East and Yorkshire     72
## 374  2020-04-25 North East and Yorkshire     69
## 375  2020-04-26 North East and Yorkshire     65
## 376  2020-04-27 North East and Yorkshire     65
## 377  2020-04-28 North East and Yorkshire     57
## 378  2020-04-29 North East and Yorkshire     69
## 379  2020-04-30 North East and Yorkshire     57
## 380  2020-05-01 North East and Yorkshire     64
## 381  2020-05-02 North East and Yorkshire     48
## 382  2020-05-03 North East and Yorkshire     40
## 383  2020-05-04 North East and Yorkshire     49
## 384  2020-05-05 North East and Yorkshire     40
## 385  2020-05-06 North East and Yorkshire     50
## 386  2020-05-07 North East and Yorkshire     45
## 387  2020-05-08 North East and Yorkshire     42
## 388  2020-05-09 North East and Yorkshire     44
## 389  2020-05-10 North East and Yorkshire     40
## 390  2020-05-11 North East and Yorkshire     29
## 391  2020-05-12 North East and Yorkshire     27
## 392  2020-05-13 North East and Yorkshire     28
## 393  2020-05-14 North East and Yorkshire     30
## 394  2020-05-15 North East and Yorkshire     32
## 395  2020-05-16 North East and Yorkshire     35
## 396  2020-05-17 North East and Yorkshire     26
## 397  2020-05-18 North East and Yorkshire     29
## 398  2020-05-19 North East and Yorkshire     27
## 399  2020-05-20 North East and Yorkshire     21
## 400  2020-05-21 North East and Yorkshire     33
## 401  2020-05-22 North East and Yorkshire     22
## 402  2020-05-23 North East and Yorkshire     18
## 403  2020-05-24 North East and Yorkshire     25
## 404  2020-05-25 North East and Yorkshire     21
## 405  2020-05-26 North East and Yorkshire     21
## 406  2020-05-27 North East and Yorkshire     21
## 407  2020-05-28 North East and Yorkshire     20
## 408  2020-05-29 North East and Yorkshire     24
## 409  2020-05-30 North East and Yorkshire     20
## 410  2020-05-31 North East and Yorkshire     19
## 411  2020-06-01 North East and Yorkshire     16
## 412  2020-06-02 North East and Yorkshire     22
## 413  2020-06-03 North East and Yorkshire     22
## 414  2020-06-04 North East and Yorkshire     17
## 415  2020-06-05 North East and Yorkshire     17
## 416  2020-06-06 North East and Yorkshire     20
## 417  2020-06-07 North East and Yorkshire     13
## 418  2020-06-08 North East and Yorkshire     11
## 419  2020-06-09 North East and Yorkshire     11
## 420  2020-06-10 North East and Yorkshire     15
## 421  2020-06-11 North East and Yorkshire      4
## 422  2020-06-12 North East and Yorkshire      8
## 423  2020-06-13 North East and Yorkshire      5
## 424  2020-06-14 North East and Yorkshire      1
## 425  2020-03-01               North West      0
## 426  2020-03-02               North West      0
## 427  2020-03-03               North West      0
## 428  2020-03-04               North West      0
## 429  2020-03-05               North West      1
## 430  2020-03-06               North West      0
## 431  2020-03-07               North West      0
## 432  2020-03-08               North West      1
## 433  2020-03-09               North West      0
## 434  2020-03-10               North West      0
## 435  2020-03-11               North West      0
## 436  2020-03-12               North West      2
## 437  2020-03-13               North West      3
## 438  2020-03-14               North West      1
## 439  2020-03-15               North West      4
## 440  2020-03-16               North West      2
## 441  2020-03-17               North West      4
## 442  2020-03-18               North West      6
## 443  2020-03-19               North West      7
## 444  2020-03-20               North West     10
## 445  2020-03-21               North West     11
## 446  2020-03-22               North West     13
## 447  2020-03-23               North West     16
## 448  2020-03-24               North West     21
## 449  2020-03-25               North West     21
## 450  2020-03-26               North West     29
## 451  2020-03-27               North West     35
## 452  2020-03-28               North West     28
## 453  2020-03-29               North West     46
## 454  2020-03-30               North West     67
## 455  2020-03-31               North West     52
## 456  2020-04-01               North West     86
## 457  2020-04-02               North West     96
## 458  2020-04-03               North West     95
## 459  2020-04-04               North West     98
## 460  2020-04-05               North West    102
## 461  2020-04-06               North West    100
## 462  2020-04-07               North West    134
## 463  2020-04-08               North West    127
## 464  2020-04-09               North West    119
## 465  2020-04-10               North West    117
## 466  2020-04-11               North West    139
## 467  2020-04-12               North West    126
## 468  2020-04-13               North West    129
## 469  2020-04-14               North West    131
## 470  2020-04-15               North West    114
## 471  2020-04-16               North West    134
## 472  2020-04-17               North West     98
## 473  2020-04-18               North West    113
## 474  2020-04-19               North West     71
## 475  2020-04-20               North West     83
## 476  2020-04-21               North West     76
## 477  2020-04-22               North West     86
## 478  2020-04-23               North West     85
## 479  2020-04-24               North West     66
## 480  2020-04-25               North West     65
## 481  2020-04-26               North West     55
## 482  2020-04-27               North West     54
## 483  2020-04-28               North West     57
## 484  2020-04-29               North West     62
## 485  2020-04-30               North West     59
## 486  2020-05-01               North West     45
## 487  2020-05-02               North West     56
## 488  2020-05-03               North West     55
## 489  2020-05-04               North West     48
## 490  2020-05-05               North West     48
## 491  2020-05-06               North West     44
## 492  2020-05-07               North West     49
## 493  2020-05-08               North West     42
## 494  2020-05-09               North West     30
## 495  2020-05-10               North West     41
## 496  2020-05-11               North West     34
## 497  2020-05-12               North West     38
## 498  2020-05-13               North West     25
## 499  2020-05-14               North West     26
## 500  2020-05-15               North West     33
## 501  2020-05-16               North West     32
## 502  2020-05-17               North West     24
## 503  2020-05-18               North West     31
## 504  2020-05-19               North West     35
## 505  2020-05-20               North West     27
## 506  2020-05-21               North West     26
## 507  2020-05-22               North West     26
## 508  2020-05-23               North West     31
## 509  2020-05-24               North West     26
## 510  2020-05-25               North West     31
## 511  2020-05-26               North West     27
## 512  2020-05-27               North West     27
## 513  2020-05-28               North West     28
## 514  2020-05-29               North West     20
## 515  2020-05-30               North West     17
## 516  2020-05-31               North West     13
## 517  2020-06-01               North West     12
## 518  2020-06-02               North West     27
## 519  2020-06-03               North West     21
## 520  2020-06-04               North West     20
## 521  2020-06-05               North West     15
## 522  2020-06-06               North West     23
## 523  2020-06-07               North West     17
## 524  2020-06-08               North West     19
## 525  2020-06-09               North West     15
## 526  2020-06-10               North West     12
## 527  2020-06-11               North West     14
## 528  2020-06-12               North West      4
## 529  2020-06-13               North West      3
## 530  2020-06-14               North West      2
## 531  2020-03-01               South East      0
## 532  2020-03-02               South East      0
## 533  2020-03-03               South East      1
## 534  2020-03-04               South East      0
## 535  2020-03-05               South East      1
## 536  2020-03-06               South East      0
## 537  2020-03-07               South East      0
## 538  2020-03-08               South East      1
## 539  2020-03-09               South East      1
## 540  2020-03-10               South East      1
## 541  2020-03-11               South East      1
## 542  2020-03-12               South East      0
## 543  2020-03-13               South East      1
## 544  2020-03-14               South East      1
## 545  2020-03-15               South East      5
## 546  2020-03-16               South East      8
## 547  2020-03-17               South East      7
## 548  2020-03-18               South East     10
## 549  2020-03-19               South East      9
## 550  2020-03-20               South East     14
## 551  2020-03-21               South East      7
## 552  2020-03-22               South East     25
## 553  2020-03-23               South East     20
## 554  2020-03-24               South East     22
## 555  2020-03-25               South East     29
## 556  2020-03-26               South East     34
## 557  2020-03-27               South East     34
## 558  2020-03-28               South East     36
## 559  2020-03-29               South East     54
## 560  2020-03-30               South East     58
## 561  2020-03-31               South East     65
## 562  2020-04-01               South East     66
## 563  2020-04-02               South East     55
## 564  2020-04-03               South East     72
## 565  2020-04-04               South East     80
## 566  2020-04-05               South East     82
## 567  2020-04-06               South East     88
## 568  2020-04-07               South East    100
## 569  2020-04-08               South East     83
## 570  2020-04-09               South East    104
## 571  2020-04-10               South East     88
## 572  2020-04-11               South East     88
## 573  2020-04-12               South East     88
## 574  2020-04-13               South East     84
## 575  2020-04-14               South East     65
## 576  2020-04-15               South East     72
## 577  2020-04-16               South East     56
## 578  2020-04-17               South East     86
## 579  2020-04-18               South East     57
## 580  2020-04-19               South East     70
## 581  2020-04-20               South East     86
## 582  2020-04-21               South East     50
## 583  2020-04-22               South East     54
## 584  2020-04-23               South East     57
## 585  2020-04-24               South East     64
## 586  2020-04-25               South East     51
## 587  2020-04-26               South East     51
## 588  2020-04-27               South East     40
## 589  2020-04-28               South East     40
## 590  2020-04-29               South East     47
## 591  2020-04-30               South East     29
## 592  2020-05-01               South East     37
## 593  2020-05-02               South East     36
## 594  2020-05-03               South East     17
## 595  2020-05-04               South East     35
## 596  2020-05-05               South East     29
## 597  2020-05-06               South East     25
## 598  2020-05-07               South East     27
## 599  2020-05-08               South East     26
## 600  2020-05-09               South East     28
## 601  2020-05-10               South East     19
## 602  2020-05-11               South East     25
## 603  2020-05-12               South East     27
## 604  2020-05-13               South East     18
## 605  2020-05-14               South East     32
## 606  2020-05-15               South East     24
## 607  2020-05-16               South East     22
## 608  2020-05-17               South East     18
## 609  2020-05-18               South East     22
## 610  2020-05-19               South East     12
## 611  2020-05-20               South East     22
## 612  2020-05-21               South East     14
## 613  2020-05-22               South East     17
## 614  2020-05-23               South East     21
## 615  2020-05-24               South East     16
## 616  2020-05-25               South East     13
## 617  2020-05-26               South East     19
## 618  2020-05-27               South East     17
## 619  2020-05-28               South East     12
## 620  2020-05-29               South East     18
## 621  2020-05-30               South East      8
## 622  2020-05-31               South East     10
## 623  2020-06-01               South East     11
## 624  2020-06-02               South East     12
## 625  2020-06-03               South East     17
## 626  2020-06-04               South East     11
## 627  2020-06-05               South East      9
## 628  2020-06-06               South East      9
## 629  2020-06-07               South East     11
## 630  2020-06-08               South East      5
## 631  2020-06-09               South East      9
## 632  2020-06-10               South East      8
## 633  2020-06-11               South East      3
## 634  2020-06-12               South East      5
## 635  2020-06-13               South East      1
## 636  2020-06-14               South East      1
## 637  2020-03-01               South West      0
## 638  2020-03-02               South West      0
## 639  2020-03-03               South West      0
## 640  2020-03-04               South West      0
## 641  2020-03-05               South West      0
## 642  2020-03-06               South West      0
## 643  2020-03-07               South West      0
## 644  2020-03-08               South West      0
## 645  2020-03-09               South West      0
## 646  2020-03-10               South West      0
## 647  2020-03-11               South West      1
## 648  2020-03-12               South West      0
## 649  2020-03-13               South West      0
## 650  2020-03-14               South West      1
## 651  2020-03-15               South West      0
## 652  2020-03-16               South West      0
## 653  2020-03-17               South West      2
## 654  2020-03-18               South West      2
## 655  2020-03-19               South West      5
## 656  2020-03-20               South West      3
## 657  2020-03-21               South West      6
## 658  2020-03-22               South West      9
## 659  2020-03-23               South West      9
## 660  2020-03-24               South West      7
## 661  2020-03-25               South West      9
## 662  2020-03-26               South West     11
## 663  2020-03-27               South West     13
## 664  2020-03-28               South West     21
## 665  2020-03-29               South West     18
## 666  2020-03-30               South West     23
## 667  2020-03-31               South West     23
## 668  2020-04-01               South West     22
## 669  2020-04-02               South West     23
## 670  2020-04-03               South West     30
## 671  2020-04-04               South West     42
## 672  2020-04-05               South West     32
## 673  2020-04-06               South West     34
## 674  2020-04-07               South West     39
## 675  2020-04-08               South West     47
## 676  2020-04-09               South West     24
## 677  2020-04-10               South West     46
## 678  2020-04-11               South West     43
## 679  2020-04-12               South West     23
## 680  2020-04-13               South West     27
## 681  2020-04-14               South West     24
## 682  2020-04-15               South West     32
## 683  2020-04-16               South West     29
## 684  2020-04-17               South West     33
## 685  2020-04-18               South West     25
## 686  2020-04-19               South West     31
## 687  2020-04-20               South West     26
## 688  2020-04-21               South West     26
## 689  2020-04-22               South West     23
## 690  2020-04-23               South West     17
## 691  2020-04-24               South West     19
## 692  2020-04-25               South West     15
## 693  2020-04-26               South West     27
## 694  2020-04-27               South West     13
## 695  2020-04-28               South West     17
## 696  2020-04-29               South West     15
## 697  2020-04-30               South West     26
## 698  2020-05-01               South West      6
## 699  2020-05-02               South West      7
## 700  2020-05-03               South West     10
## 701  2020-05-04               South West     17
## 702  2020-05-05               South West     14
## 703  2020-05-06               South West     19
## 704  2020-05-07               South West     16
## 705  2020-05-08               South West      6
## 706  2020-05-09               South West     11
## 707  2020-05-10               South West      5
## 708  2020-05-11               South West      8
## 709  2020-05-12               South West      7
## 710  2020-05-13               South West      7
## 711  2020-05-14               South West      6
## 712  2020-05-15               South West      4
## 713  2020-05-16               South West      4
## 714  2020-05-17               South West      6
## 715  2020-05-18               South West      4
## 716  2020-05-19               South West      6
## 717  2020-05-20               South West      1
## 718  2020-05-21               South West      9
## 719  2020-05-22               South West      6
## 720  2020-05-23               South West      6
## 721  2020-05-24               South West      3
## 722  2020-05-25               South West      8
## 723  2020-05-26               South West     11
## 724  2020-05-27               South West      5
## 725  2020-05-28               South West      9
## 726  2020-05-29               South West      6
## 727  2020-05-30               South West      3
## 728  2020-05-31               South West      2
## 729  2020-06-01               South West      6
## 730  2020-06-02               South West      2
## 731  2020-06-03               South West      5
## 732  2020-06-04               South West      2
## 733  2020-06-05               South West      2
## 734  2020-06-06               South West      1
## 735  2020-06-07               South West      3
## 736  2020-06-08               South West      3
## 737  2020-06-09               South West      0
## 738  2020-06-10               South West      0
## 739  2020-06-11               South West      2
## 740  2020-06-12               South West      2
## 741  2020-06-13               South West      1
## 742  2020-06-14               South West      0

1.5 Completion date

We extract the completion date from the NHS Pathways file timestamp:


database_date <- attr(x, "timestamp")
database_date
## [1] "2020-06-15"

The completion date of the NHS Pathways data is Monday 15 Jun 2020.

1.6 Auxiliary functions

These are functions which will be used further in the analyses.

Function to estimate the generalised R-squared as the proportion of deviance explained by a given model:


## Function to calculate R2 for Poisson model
## not adjusted for model complexity but all models have the same DF here

Rsq <- function(x) {
  1 - (x$deviance / x$null.deviance)
}

Function to extract growth rates per region as well as halving times, and the associated 95% confidence intervals:


## function to extract the coefficients, find the level of the intercept,
## reconstruct the values of r, get confidence intervals

get_r <- function(model) {
  ##  extract coefficients and conf int
  out <- data.frame(r = coef(model))  %>%
    rownames_to_column("var") %>% 
    cbind(confint(model)) %>%
    filter(!grepl("day_of_week", var)) %>% 
    filter(grepl("day", var)) %>%
    rename(lower_95 = "2.5 %",
           upper_95 = "97.5 %") %>%
    mutate(var = sub("day:", "", var))
  
  ## reconstruct values: intercept + region-coefficient
  for (i in 2:nrow(out)) {
    out[i, -1] <- out[1, -1] + out[i, -1]
  }
  
  ## find the name of the intercept, restore regions names
  out <- out %>%
    mutate(nhs_region = model$xlevels$nhs_region) %>%
    select(nhs_region, everything(), -var)
  
  ## find halving times
  halving <- log(0.5) / out[,-1] %>%
    rename(halving_t = r,
           halving_t_lower_95 = lower_95,
           halving_t_upper_95 = upper_95)
  
  ## set halving times with exclusion intervals to NA
  no_halving <- out$lower_95 < 0 & out$upper_95 > 0
  halving[no_halving, ] <- NA_real_
  
  ## return all data
  cbind(out, halving)
  
}

Functions used in the correlation analysis between NHS Pathways reports and deaths:

## Function to calculate Pearson's correlation between deaths and lagged
## reports. Note that `pearson` can be replaced with `spearman` for rank
## correlation.

getcor <- function(x, ndx) {
  return(cor(x$deaths[ndx],
             x$note_lag[ndx],
             use = "complete.obs",
             method = "pearson"))
}

## Catch if sample size throws an error
getcor2 <- possibly(getcor, otherwise = NA)

getboot <- function(x) {
  result <- boot::boot.ci(boot::boot(x, getcor2, R = 1000), 
                           type = "bca")
  return(data.frame(n = sum(!is.na(x$note_lag) & !is.na(x$deaths)),
                    r = result$t0,
                    r_low = result$bca[4],
                    r_hi = result$bca[5]))
}

Function to classify the day of the week into weekend, Monday, and the rest:


## Fn to add day of week
day_of_week <- function(df) {
  df %>% 
    dplyr::mutate(day_of_week = lubridate::wday(date, label = TRUE)) %>% 
    dplyr::mutate(day_of_week = dplyr::case_when(
      day_of_week %in% c("Sat", "Sun") ~ "weekend",
      day_of_week %in% c("Mon") ~ "monday",
      !(day_of_week %in% c("Sat", "Sun", "Mon")) ~ "rest_of_week"
    ) %>% 
      factor(levels = c("rest_of_week", "monday", "weekend")))
}

Custom color palettes, color scales, and vectors of colors:


pal <- c("#006212",
         "#ae3cab",
         "#00db90",
         "#960c00",
         "#55aaff",
         "#ff7e78",
         "#00388d")

age.pal <- viridis::viridis(3,begin = 0.1, end = 0.7)

3 Comparison with deaths time series

3.1 Outline

We want to explore the correlation between NHS Pathways reports and deaths, and assess the potential for reports to be used as an early warning system for disease resurgence.

Death data are publically available. We truncate the time series to avoid bias from reporting delay - we assume a conservative delay of three weeks.

3.2 Lagged correlation

We calculate Pearson’s correlation coefficient between deaths and NHS Pathways notifications using different lags. Confidence intervals are obtained using bootstrap. Note that results were also confirmed using Spearman’s rank correlation.

First we join the NHS Pathways and death data, and aggregate over all England:

## truncate death data for reporting delay
trunc_date <- max(dth$date_report) - delay_max

dth_trunc <- dth %>%
  rename(date = date_report) %>%
  filter(date <= trunc_date) 

## join with notification data
all_data <- x %>% 
  filter(!is.na(nhs_region)) %>%
  group_by(date, nhs_region) %>%
  summarise(count = sum(count, na.rm = T)) %>%
  ungroup %>%
  inner_join(dth_trunc,
             by = c("date","nhs_region"))

all_tot <- all_data %>%
  group_by(date) %>%
  summarise(count = sum(count, na.rm = TRUE),
            deaths = sum(deaths, na.rm = TRUE)) 

We calculate correlation with lagged NHS Pathways reports from 0 to 30 days behind deaths:


## Calculate all correlations + bootstrap CIs
lag_cor <- data.frame()
for (i in 0:30) {
  
  ## lag reports
  summary <- all_tot %>% 
    mutate(note_lag = lag(count, i)) %>%
    ## calculate rank correlation and bootstrap CI
    getboot(.) %>%
    mutate(lag = i)

  lag_cor <- bind_rows(lag_cor, summary)
}

cor_vs_lag <- ggplot(lag_cor, aes(lag, r)) +
  theme_bw() +
  geom_ribbon(aes(ymin = r_low, ymax = r_hi), alpha = 0.2) +
  geom_hline(yintercept = 0, lty = "longdash") +
  geom_point() +
  geom_line() +
  labs(x = "Lag between NHS pathways and death data (days)",
       y = "Pearson's correlation") +
  large_txt
cor_vs_lag


l_opt <- which.max(lag_cor$r)

This analysis suggests that the best lag is 23 days. We then compare and plot the number of deaths reported against the number of NHS Pathways reports lagged by 23 days.


all_tot <- all_tot %>%
  rename(date_death = date) %>%
  mutate(note_lag = lag(count, lag_cor$lag[l_opt]),
         note_lag_c = (note_lag - mean(note_lag, na.rm = T)),
         date_note = lag(date_death,16))

lag_mod <- glm(deaths ~ note_lag, data = all_tot, family = "quasipoisson")

summary(lag_mod)
## 
## Call:
## glm(formula = deaths ~ note_lag, family = "quasipoisson", data = all_tot)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -8.8946  -2.2215  -0.3088   2.5363   4.4960  
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 5.000e+00  5.317e-02   94.02   <2e-16 ***
## note_lag    1.120e-05  5.240e-07   21.38   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for quasipoisson family taken to be 10.08451)
## 
##     Null deviance: 4980.7  on 44  degrees of freedom
## Residual deviance:  448.8  on 43  degrees of freedom
##   (23 observations deleted due to missingness)
## AIC: NA
## 
## Number of Fisher Scoring iterations: 4

exp(coefficients(lag_mod))
## (Intercept)    note_lag 
##  148.351112    1.000011
exp(confint(lag_mod))
##                 2.5 %     97.5 %
## (Intercept) 133.51866 164.466487
## note_lag      1.00001   1.000012

Rsq(lag_mod)
## [1] 0.9098908

mod_fit <- as.data.frame(predict(lag_mod, type = "link", se.fit = TRUE)[1:2])

all_tot_pred <- 
  all_tot %>%
  filter(!is.na(note_lag)) %>%
  mutate(pred = mod_fit$fit,
         pred.se = mod_fit$se.fit,
         low = exp(pred - 1.96*pred.se),
         hi = exp(pred + 1.96*pred.se))


glm_fit <- all_tot_pred %>% 
    filter(!is.na(note_lag)) %>%
  ggplot(aes(x = note_lag, y = deaths)) +
  geom_point() + 
  geom_line(aes(y = exp(pred))) + 
  geom_ribbon(aes(ymin = low, ymax = hi), alpha = 0.3, col = "grey") +
  theme_bw() +
  labs(y = "Daily number of\ndeaths reported",
       x = "Daily number of NHS Pathways reports") +
  large_txt

glm_fit

4 Supplementary figures

4.1 Serial interval distribution

This is a comparison of gamma versus lognormal distribution for the serial interval used to convert r to R in our analysis. Both distributions are parameterised with mean 4.7 and standard deviation 2.9.

SI_param <- epitrix::gamma_mucv2shapescale(4.7, 2.9/4.7)
SI_distribution <- distcrete::distcrete("gamma", interval = 1,
                                        shape = SI_param$shape,
                                        scale = SI_param$scale, w = 0.5)

SI_distribution2 <- distcrete::distcrete("lnorm", interval = 1,
                                        meanlog = log(4.7),
                                        sdlog = log(2.9), w = 0.5)

SI_dist1 <- data.frame(x = SI_distribution$r(1e5)) 
SI_dist1 <- count(SI_dist1, x) %>%
    ggplot() +
    geom_col(aes(x = x, y = n)) +
    labs(x = "Serial interval (days)", y = "Frequency") +
    scale_x_continuous(breaks = seq(0, 30, 5)) +
    theme_bw()

SI_dist2 <- data.frame(x = SI_distribution2$r(1e5)) 
SI_dist2 <- count(SI_dist2, x) %>%
    ggplot() +
    geom_col(aes(x = x, y = n)) +
    labs(x = "Serial interval (days)", y = "Frequency") +
    scale_x_continuous(breaks = seq(0, 200, 20), limits = c(0, 200)) +
    theme_bw()


ggpubr::ggarrange(SI_dist1,
                  SI_dist2,
                  nrow = 1,
                  labels = "AUTO") 

4.2 Sensitivity analysis - 7 or 21 days moving window

We reproduce the window analysis with either a 7 or 21 days window for sensitivity purposes.

First with the 7 days window:

## set moving time window (1/2/3 weeks)
w <- 7

# create empty df
r_all_sliding_7days <- NULL

## make data for model
x_model_all_moving <- x %>%
  filter(!is.na(nhs_region)) %>% 
  group_by(date, nhs_region) %>%
  summarise(n = sum(count)) 

unique_dates <- unique(x_model_all_moving$date)

for (i in 1:(length(unique_dates) - w)) {
  
  date_i <- unique_dates[i]
  
  date_i_max <- date_i + w
  
  model_data <- x_model_all_moving %>%
    filter(date >= date_i & date < date_i_max) %>%
    mutate(day = as.integer(date - date_i)) %>% 
    day_of_week()
  
  
  mod <- glm(n ~ day * nhs_region + day_of_week,
             data = model_data,
             family = 'quasipoisson')
  
  # get growth rate
  r <- get_r(mod)
  r$w_min <- date_i
  r$w_max <- date_i_max
  
  # combine all estimates
  r_all_sliding_7days <- bind_rows(r_all_sliding_7days, r)
  
}

#serial interval distribution
SI_param = epitrix::gamma_mucv2shapescale(4.7, 2.9/4.7)
SI_distribution <- distcrete::distcrete("gamma", interval = 1,
                                        shape = SI_param$shape,
                                        scale = SI_param$scale,
                                        w = 0.5)

#convert growth rates r to R0
r_all_sliding_7days <- r_all_sliding_7days %>%
  mutate(R = epitrix::r2R0(r, SI_distribution),
         R_lower_95 = epitrix::r2R0(lower_95, SI_distribution),
         R_upper_95 = epitrix::r2R0(upper_95, SI_distribution))
# plot
plot_growth <-
  r_all_sliding_7days %>%
  ggplot(aes(x = w_max, y = r)) +
  geom_ribbon(aes(ymin = lower_95, ymax = upper_95, fill = nhs_region), alpha = 0.1) +
  geom_line(aes(colour = nhs_region)) +
  geom_point(aes(colour = nhs_region)) +
  geom_hline(yintercept = 0, linetype = "dashed") +
  theme_bw() +
  scale_weeks +
  theme(legend.position = "bottom",
        plot.margin = margin(0.5,1,0.5,0.5, "cm")) +
  guides(colour = guide_legend(title = "",override.aes = list(fill = NA)), fill = FALSE) +
  labs(x = "",
       y = "Estimated daily growth rate (r)") +
  scale_colour_manual(values = pal)
plot_R <- r_all_sliding_7days %>%
  ggplot(aes(x = w_max, y = R)) +
  geom_ribbon(aes(ymin = R_lower_95, ymax = R_upper_95, fill = nhs_region), alpha = 0.1) +
  geom_line(aes(colour = nhs_region)) +
  geom_point(aes(colour = nhs_region)) +
  geom_hline(yintercept = 1, linetype = "dashed") +
  theme_bw() +
  scale_weeks +
  theme(legend.position = "bottom",
        plot.margin = margin(0.5,1,0.5,0.5, "cm")) +
  guides(color = guide_legend(title = "", override.aes = list(fill = NA)), fill = FALSE) +
  labs(x = "",
       y = "Estimated effective reproduction\nnumber (Re)") +
  scale_colour_manual(values = pal)

R <- r_all_sliding_7days %>%
  mutate(lower_95 = R_lower_95, 
         upper_95 = R_upper_95,
         value = R,
         measure = "R",
         reference = 1)

r_R <- r_all_sliding_7days %>%
  mutate(measure = "r",
         value = r,
         reference = 0) %>%
  bind_rows(R)

r_R_7 <- r_R %>%
  ggplot(aes(x = w_max, y = value)) +
  geom_ribbon(aes(ymin = lower_95, ymax = upper_95, fill = nhs_region), alpha = 0.1) +
  geom_line(aes(colour = nhs_region)) +
  geom_point(aes(colour = nhs_region)) +
  geom_hline(aes(yintercept = reference), linetype = "dashed") +
  theme_bw() +
  scale_weeks +
  theme(legend.position = "bottom",
        plot.margin = margin(0.5,1,0,0, "cm"),
        strip.background = element_blank(),
        strip.placement = "outside"
  ) +
  guides(color = guide_legend(title = "", override.aes = list(fill = NA)), fill = FALSE) +
  labs(x = "", y = "") +
  scale_colour_manual(values = pal) +
  facet_grid(rows = vars(measure),
             scales = "free_y",
             switch = "y",
             labeller = as_labeller(c(r = "Daily growth rate (r)",
                                      R = "Effective reproduction\nnumber (Re)")))

Then with the 21 days window:

## set moving time window (1/2/3 weeks)
w <- 21

# create empty df
r_all_sliding_21days <- NULL

## make data for model
x_model_all_moving <- x %>%
  filter(!is.na(nhs_region)) %>% 
  group_by(date, nhs_region) %>%
  summarise(n = sum(count)) 

unique_dates <- unique(x_model_all_moving$date)

for (i in 1:(length(unique_dates) - w)) {
  
  date_i <- unique_dates[i]
  
  date_i_max <- date_i + w
  
  model_data <- x_model_all_moving %>%
    filter(date >= date_i & date < date_i_max) %>%
    mutate(day = as.integer(date - date_i)) %>% 
    day_of_week()
  
  
  mod <- glm(n ~ day * nhs_region + day_of_week,
             data = model_data,
             family = 'quasipoisson')
  
  # get growth rate
  r <- get_r(mod)
  r$w_min <- date_i
  r$w_max <- date_i_max
  
  # combine all estimates
  r_all_sliding_21days <- bind_rows(r_all_sliding_21days, r)
  
}

#serial interval distribution
SI_param = epitrix::gamma_mucv2shapescale(4.7, 2.9/4.7)
SI_distribution <- distcrete::distcrete("gamma", interval = 1,
                                        shape = SI_param$shape,
                                        scale = SI_param$scale,
                                        w = 0.5)

#convert growth rates r to R0
r_all_sliding_21days <- r_all_sliding_21days %>%
  mutate(R = epitrix::r2R0(r, SI_distribution),
         R_lower_95 = epitrix::r2R0(lower_95, SI_distribution),
         R_upper_95 = epitrix::r2R0(upper_95, SI_distribution))
# plot
plot_growth <-
  r_all_sliding_21days %>%
  ggplot(aes(x = w_max, y = r)) +
  geom_ribbon(aes(ymin = lower_95, ymax = upper_95, fill = nhs_region), alpha = 0.1) +
  geom_line(aes(colour = nhs_region)) +
  geom_point(aes(colour = nhs_region)) +
  geom_hline(yintercept = 0, linetype = "dashed") +
  theme_bw() +
  scale_weeks +
  theme(legend.position = "bottom",
        plot.margin = margin(0.5,1,0.5,0.5, "cm")) +
  guides(colour = guide_legend(title = "",override.aes = list(fill = NA)), fill = FALSE) +
  labs(x = "",
       y = "Estimated daily growth rate (r)") +
  scale_colour_manual(values = pal)
# plot
plot_R <-
  r_all_sliding_21days %>%
  ggplot(aes(x = w_max, y = R)) +
  geom_ribbon(aes(ymin = R_lower_95, ymax = R_upper_95, fill = nhs_region), alpha = 0.1) +
  geom_line(aes(colour = nhs_region)) +
  geom_point(aes(colour = nhs_region)) +
  geom_hline(yintercept = 1, linetype = "dashed") +
  theme_bw() +
  scale_weeks +
  theme(legend.position = "bottom",
        plot.margin = margin(0.5,1,0.5,0.5, "cm")) +
  guides(color = guide_legend(title = "", override.aes = list(fill = NA)), fill = FALSE) +
  labs(x = "",
       y = "Estimated effective reproduction\nnumber (Re)") +
  scale_colour_manual(values = pal)

R <- r_all_sliding_21days %>%
  mutate(lower_95 = R_lower_95, 
         upper_95 = R_upper_95,
         value = R,
         measure = "R",
         reference = 1)

r_R <- r_all_sliding_21days %>%
  mutate(measure = "r",
         value = r,
         reference = 0) %>%
  bind_rows(R)

r_R_21 <- r_R %>%
  ggplot(aes(x = w_max, y = value)) +
  geom_ribbon(aes(ymin = lower_95, ymax = upper_95, fill = nhs_region), alpha = 0.1) +
  geom_line(aes(colour = nhs_region)) +
  geom_point(aes(colour = nhs_region)) +
  geom_hline(aes(yintercept = reference), linetype = "dashed") +
  theme_bw() +
  scale_weeks +
  theme(legend.position = "bottom",
        plot.margin = margin(0.5,1,0,0, "cm"),
        strip.background = element_blank(),
        strip.placement = "outside"
  ) +
  guides(color = guide_legend(title = "", override.aes = list(fill = NA)), fill = FALSE) +
  labs(x = "", y = "") +
  scale_colour_manual(values = pal) +
  facet_grid(rows = vars(measure),
             scales = "free_y",
             switch = "y",
             labeller = as_labeller(c(r = "Daily growth rate (r)",
                                      R = "Effective reproduction\nnumber (Re)")))

And we combine both outputs into a single plot:


ggpubr::ggarrange(r_R_7,
                  r_R_21,
                  nrow = 2,
                  labels = "AUTO",
                  common.legend = TRUE,
                  legend = "bottom") 

4.3 Correlation between NHS Pathways reports and deaths by NHS region


lag_cor_reg <- data.frame()

for (i in 0:30) {

  summary <-
    all_data %>%
    group_by(nhs_region) %>%
    mutate(note_lag = lag(count, i)) %>%
    ## calculate rank correlation and bootstrap CI for each region
    group_modify(~getboot(.x)) %>%
    mutate(lag = i)
  
  lag_cor_reg <- bind_rows(lag_cor_reg, summary)
}

cor_vs_lag_reg <- 
lag_cor_reg %>%
ggplot(aes(lag, r, col = nhs_region)) +
  geom_hline(yintercept = 0, lty = "longdash") +
  geom_ribbon(aes(ymin = r_low, ymax = r_hi, col = NULL, fill = nhs_region), alpha = 0.2) +
  geom_point() +
  geom_line() +
  facet_wrap(~nhs_region) +
  scale_color_manual(values = pal) +
  scale_fill_manual(values = pal, guide = F) +  
  theme_bw() +
  labs(x = "Lag between NHS pathways and death data (days)", y = "Pearson's correlation", col = "NHS region") +
  theme(legend.position = "bottom") +
  guides(color = guide_legend(override.aes = list(fill = NA)))

cor_vs_lag_reg

5 Export data

We save the tables created during our analysis:


if (!dir.exists("excel_tables")) {
  dir.create("excel_tables")
}


## list all tables, and loop over export
tables_to_export <- c("r_all_sliding", "lag_cor")

for (e in tables_to_export) {
  rio::export(get(e),
              file.path("excel_tables",
                        paste0(e, ".xlsx")))
}

## also export result from regression on lagged data 
rio::export(lag_mod, file.path("excel_tables", "lag_mod.rds"))

6 System information

6.1 Outline

The following information documents the system on which the document was compiled.

6.2 System

This provides information on the operating system.

Sys.info()
##                                                                                            sysname 
##                                                                                           "Darwin" 
##                                                                                            release 
##                                                                                           "19.5.0" 
##                                                                                            version 
## "Darwin Kernel Version 19.5.0: Tue May 26 20:41:44 PDT 2020; root:xnu-6153.121.2~2/RELEASE_X86_64" 
##                                                                                           nodename 
##                                                                                   "Mac-1467.local" 
##                                                                                            machine 
##                                                                                           "x86_64" 
##                                                                                              login 
##                                                                                             "root" 
##                                                                                               user 
##                                                                                           "runner" 
##                                                                                     effective_user 
##                                                                                           "runner"

6.3 R environment

This provides information on the version of R used:

R.version
##                _                           
## platform       x86_64-apple-darwin15.6.0   
## arch           x86_64                      
## os             darwin15.6.0                
## system         x86_64, darwin15.6.0        
## status                                     
## major          3                           
## minor          6.3                         
## year           2020                        
## month          02                          
## day            29                          
## svn rev        77875                       
## language       R                           
## version.string R version 3.6.3 (2020-02-29)
## nickname       Holding the Windsock

6.4 R packages

This provides information on the packages used:

sessionInfo()
## R version 3.6.3 (2020-02-29)
## Platform: x86_64-apple-darwin15.6.0 (64-bit)
## Running under: macOS Catalina 10.15.5
## 
## Matrix products: default
## BLAS:   /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRblas.0.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib
## 
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] ggnewscale_0.4.1     ggpubr_0.3.0         lubridate_1.7.9     
##  [4] chngpt_2020.5-21     cyphr_1.1.0          DT_0.13             
##  [7] kableExtra_1.1.0     janitor_2.0.1        remotes_2.1.1       
## [10] projections_0.4.1    earlyR_0.0.1         epitrix_0.2.2       
## [13] distcrete_1.0.3      incidence_1.7.1      rio_0.5.16          
## [16] reshape2_1.4.4       rvest_0.3.5          xml2_1.3.2          
## [19] linelist_0.0.40.9000 forcats_0.5.0        stringr_1.4.0       
## [22] dplyr_1.0.0          purrr_0.3.4          readr_1.3.1         
## [25] tidyr_1.1.0          tibble_3.0.1         ggplot2_3.3.1       
## [28] tidyverse_1.3.0      here_0.1             reportfactory_0.0.5 
## 
## loaded via a namespace (and not attached):
##  [1] colorspace_1.4-1  selectr_0.4-2     ggsignif_0.6.0    ellipsis_0.3.1   
##  [5] rprojroot_1.3-2   snakecase_0.11.0  fs_1.4.1          rstudioapi_0.11  
##  [9] farver_2.0.3      fansi_0.4.1       splines_3.6.3     knitr_1.28       
## [13] jsonlite_1.6.1    broom_0.5.6       dbplyr_1.4.4      compiler_3.6.3   
## [17] httr_1.4.1        backports_1.1.7   assertthat_0.2.1  Matrix_1.2-18    
## [21] cli_2.0.2         htmltools_0.4.0   prettyunits_1.1.1 tools_3.6.3      
## [25] gtable_0.3.0      glue_1.4.1        Rcpp_1.0.4.6      carData_3.0-4    
## [29] cellranger_1.1.0  vctrs_0.3.1       nlme_3.1-144      matchmaker_0.1.1 
## [33] crosstalk_1.1.0.1 xfun_0.14         ps_1.3.3          openxlsx_4.1.5   
## [37] lifecycle_0.2.0   rstatix_0.5.0     MASS_7.3-51.5     scales_1.1.1     
## [41] hms_0.5.3         sodium_1.1        yaml_2.2.1        curl_4.3         
## [45] gridExtra_2.3     stringi_1.4.6     kyotil_2019.11-22 boot_1.3-24      
## [49] pkgbuild_1.0.8    zip_2.0.4         rlang_0.4.6       pkgconfig_2.0.3  
## [53] evaluate_0.14     lattice_0.20-38   labeling_0.3      htmlwidgets_1.5.1
## [57] cowplot_1.0.0     processx_3.4.2    tidyselect_1.1.0  plyr_1.8.6       
## [61] magrittr_1.5      R6_2.4.1          generics_0.0.2    DBI_1.1.0        
## [65] pillar_1.4.4      haven_2.3.1       foreign_0.8-75    withr_2.2.0      
## [69] mgcv_1.8-31       survival_3.1-8    abind_1.4-5       modelr_0.1.8     
## [73] crayon_1.3.4      car_3.0-8         utf8_1.1.4        rmarkdown_2.2    
## [77] viridis_0.5.1     grid_3.6.3        readxl_1.3.1      data.table_1.12.8
## [81] blob_1.2.1        callr_3.4.3       reprex_0.3.0      digest_0.6.25    
## [85] webshot_0.5.2     munsell_0.5.0     viridisLite_0.3.0